Distributed path planning for mobile sensors

ABSTRACT

A method plans paths of a set of mobile sensors with changeable positions and orientations in an environment. Each sensor includes a processor, an imaging system and a communication system. A desired resolution of coverage of the environment is defined, and an achieved resolution of the coverage is initialized. For each time instant and each sensor, an image of the environment is acquired using the imaging system. The achieved resolution is updated according to the image. The sensor is moved to a next position and orientation based on the achieved resolution and the desired resolution. Then, local information of the sensor is distributed to other sensors using the communication system to optimize a coverage of the environment.

FIELD OF THE INVENTION

The invention relates generally to mobile sensors, and more particularlyto distributed path planning for the sensors.

BACKGROUND OF THE INVENTION

Mobile sensor can be used to acquire images in a coordinated manner. Theimages can then be used in applications such as surveilance,cartography, and environmental monitoring. One problem in such systemsis planning the paths the sensors should follow, a problem known as pathplanning.

In one such system with holonomic robots, where the controllable degreesof freedom are equal to the total degrees of freedom, anisotropicsensors with a bounded footprint are considered, see Hexsel et al.,“Distributed Coverage Control for Mobile Anisotropic Sensor Networks,”Tech. Report CMU-RI-TR-13-01, Robotics Institute, Carnegie MellonUniversity, January 2013. That system models a 2-dimensional (2D)environment as a polygon, possibly containing obstacles. A fixedobjective function maximizes a joint probability to detect objects. Theobjective function uses an a priori fixed density function thatrepresents an importance of each point in the environment.

SUMMARY OF THE INVENTION

The embodiments of the invention provide a method for planning paths ofa set of mobile sensors, that can be, for example, airborne,ground-based, or underwater. Each sensor includes an imaging system forimaging an environment. The imaging system can use optical imaging,synthetic aperture radar (SAR), hyperspectral imaging, physical apertureradar imaging, for example. Images acquired by the sensors can be usedin surveillance, cartography and monitoring applications, among others.

At any time instant, each sensor moves to a next position andorientation to optimize a resolution of the imaging system is used toselect a resolution for optimal coverage of the environment, accordingto a pre-specified desired resolution at each point in the environment.As used herein, the resolution depends on the size and density of pixelsin the images. It should be noted that the coverage can be complete, orpartial and images can overlap or not.

In a distributed manner, the motion of each sensor and its orientationare optimized by minimizing a cost function that characterizes how wellthe environment has been imaged, compared to the pre-specifiedresolution. To perform this optimization, each sensor communicates withneighboring sensors, and exchanges local information. The imagesacquired by the sensors can be combined into an overall image of theenvironment to achieve the desired resolution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic of an example airborne mobile sensor systemaccording to embodiments of the invention;

FIG. 2 is a side view of a sensor and environment in sensor coordinatesaccording to embodiments of the invention;

FIG. 3 is a table giving variables, description, and exemplar valuesused by a model according to embodiments of the invention;

FIG. 4A is a schematic of an example environment according toembodiments of the invention;

FIG. 4B is a schematic of an irregular example environment;

FIG. 5 is a top view of the sensor footprint in sensor coordinatesaccording to embodiments of the invention; and

FIG. 6 is a flow diagram of a method for planning paths of the sensorsshown in FIG. 1 according to embodiments of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 1 shows a set of mobile sensors 100 according to embodiments of ourinvention. The sensors can be airborne, ground-based or underwater,among others. The sensors can, for example, be arranged in indoor oroutdoor drones, aircraft, or satellites. Each sensor includes aprocessor 101, an imaging system 102, and a communication system 103.The imaging system has a known footprint for imaging an environment 400,which may depend on the orientation of the sensor. The footprint is aprojection of the imaging plane, such as a camera image plane or a radarbeam pattern, onto the environment. The imaging system can use, amongothers, optical imaging, synthetic aperture radar (SAR), hyperspectralimaging, physical aperture radar imaging. For this reason, the term“image” is used broadly.

The sensors move along paths 102 to image the environment 400. Thecommunication system includes a tranceiver so that the sensors cancommunicate with each other over channels 105. In the preferredembodiment the channels are wireless using, e.g., radio or opticalsignals. By exchanging information between neighboring sensors, i.e.,sensors within communication range, the sensors can perform the pathplanning in a distrubted manner. However, it should be undertood thatthe other communication techniques can also be used, for example some orall of the sensors can use digital fine wire tethers.

It is an objective to provide an image resolution over the environmentthat achieves a specified value at each point in the environment. Asused herein, the resolution depends on the size and density of pixels inthe images. The model uses a subadditive function to model theresolution provided by overlapping footprints 104. The model also usesan objective function that is time varying. At each time j, theobjective function provides a measure of a difference between a desiredresolution and a resolution achieved up to a previous time j−1.

Sensor Model

FIG. 2 is a side view of a sensor and environment in sensor coordinates.A point z in the environment is located on a line bisecting an angleγ_(v). The table in FIG. 3 gives the variables 301, description 302 andexemplar values 303 used by our model. The variables include a height311 of the sensor, horizontal 312 and vertical 313 angular widths,position 314 of the sensor, and declination 315 and azimuth 317 angles.The angles specify the orientation of the sensors. In the describedembodiment, there are two degrees of freedom, however three degrees arenot precluded.

For conveninece, most of the subsequent computations use the angle ψ 316to measure the declination, which is related to the actual declinationangle Φ using ψ=90°−Φ. The labeled ranges in FIG. 2 are given by thefollowing:Z _(max) =H tan(ψ+γ_(v)/2)Z _(min) =H tan(ψ−γ_(v)/2)z=H tan(ψ).  (1)

As shown in FIG. 4A, an example regular environment is a 100×100polygonal region Q in the xy plane. An arbitrary point 401 in theenvironment is labeled q∈Q. In sensor coordinates, the environment isthe zy plane, which is a translated and rotated version of the xy plane.A given sensor is located at a height H above the origin of the zy planeand the angle ψ is measured with respect to the z axis. The height H,and the x,y location specify the position of the sensors. The angles Φand ψ specify the orientation.

FIG. 4B shows an irregular example irregular environment 500.

Each sensor has the declination angle Φ, with ψ=90°−Φ. When the azimuthangle θ=0, the z-axis of the sensor is aligned with the x-axis of aglobal coordinate system. All variables associated with the i^(th)sensor have a superscript i for indexing the sensors.

FIG. 5 is an example of a top view of the sensor footprint 104 in sensorcoordinates specific to an optical sensor. The footprint extends fromZ_(min) to Z_(max) as shown in FIG. 2. The footprint is a polygondefined by the four labeled vertices (1, 2, 3, 4). In sensorcoordinates, the footprint vertices (z_(k), y_(k)) corresponding to thek^(th) vertex are

$\begin{matrix}{\begin{bmatrix}z_{1} & y_{1} \\z_{2} & y_{2} \\z_{3} & y_{3} \\z_{4} & y_{4}\end{bmatrix} = {\begin{bmatrix}Z_{\min} & {{- Z_{\min}}{\sin\left( {\gamma\;{h/2}} \right)}} \\Z_{\max} & {{- Z_{\max}}{\sin\left( {\gamma\;{h/2}} \right)}} \\Z_{\max} & {Z_{\max}{\sin\left( {\gamma\;{h/2}} \right)}} \\Z_{\min} & {Z_{\min}{\sin\left( {\gamma\;{h/2}} \right)}}\end{bmatrix}.}} & (2)\end{matrix}$

Let S(θ) be the rotation matrix defined below

$\begin{matrix}{{S(\theta)} = {\begin{bmatrix}{\cos(\theta)} & {- {\sin(\theta)}} \\{\sin(\theta)} & {\cos(\theta)}\end{bmatrix}.}} & (3)\end{matrix}$

The global coordinates (x, y) of a point (z, y) in the sensor footprintare obtained by rotating the point by the azumith angle θ, andtranslating the footprint by the camera location (c_(x), c_(y)):

$\begin{matrix}{\begin{bmatrix}x \\y\end{bmatrix} = {{{S(\theta)}\begin{bmatrix}z \\y\end{bmatrix}} + {\begin{bmatrix}c_{x} \\c_{y}\end{bmatrix}.}}} & (4)\end{matrix}$

The four vertices in equation (2) defining the sensor footprint 104 canbe transformed into global coordinates using equation (4). The sensorfootprint is defined by four variables (c_(x),c_(y),θ,φ). The first twovariables are the projection of the sensor position onto the environmentplane, and the last two parameters are the horizontal and verticalangular variables. Other sensors may have different footprint shapes,with the footprint shape and size depending on the sensor orientation.

In most practical embodiments, the position parameters are updated on arelatively slow time scale because these parameters correspond to thephysical position of the sensor, while the angular variables are updatedon a relatively fast time scale because the angles can quickly changevalues. However, in some embodiments, position parameters and angleparameters might change values at the same time scale, or angleparameters might change values at a slower time scale.

Subadditive Combination of Overlapping Sensors

Assume that sensor i provides a resolution r_(i) in the footprint F_(i),i=1, . . . , n. The problem is to model the resolution obtained in anintersection of complete or partial overlapping footprints. The best,and unrealistically optimistic, situation is for the overall resolutionto be the sum of the individual resolutions. The worst, andunrealistically pessimistic, situation is for the overall resolution toequal the maximum of the individual sensor resolutions. The actualoverall resolution is somewhere between these extremes.

That is, ifr=[r ₁ r ₂ . . . r _(N)]  (5)is a vector of the resolutions achieved by N sensors, the overallresolution res(r) obtained at points in the intersection of the sensorfootprints satisfies the following inequalities

$\begin{matrix}{{\max\limits_{i}\; r_{i}} \leq {{res}(r)} \leq {r_{1} + \ldots + {r_{N}.}}} & (6)\end{matrix}$

One example of a function that satisfies this property is the l_(p) normof the vector r, 1<p<∞,

$\begin{matrix}{{{r}_{p}\overset{def}{=}\left( {r_{l}^{p} + {\ldots\mspace{14mu} r_{N}^{p}}} \right)^{1/p}},} & (7)\end{matrix}$where 1<p<∞. When p=1, the l_(p) norm equals the upper bound in equation(6). When p=∞, the l_(p) norm equals the lower bound in equation (6).Thus, a particular example of a subadditive model for the resolutionobtained by overlapping sensors is the l_(p) norm of a vector ofindividual resolutions, where 1<p<∞. Other embodiments can use differentsubadditive functions to model how images of different resolutions arecombined.

Objective Function and Optimization

Let φ_(d)(q) be the desired resolution defined at every point q 401 inthe environment 400. Let x_(j) be a vector of the position variables ofall of the sensors at time j. Let ψ_(j) and θ_(j) be vectorscorresponing to the vertical (declination) and horizontal (azimuth)angular variables at time j, respectively, of all of the sensors. LetR^(i) be the resolution provided by the i^(th) sensor at all points inits footprint F_(i), which is defined by sensor variables(cx^(i),cy^(i),θ^(i),ψ^(i)) as

$\begin{matrix}{{R^{i}\left( {{cx}^{i},{cy}^{i},\theta^{i},\psi^{i},q} \right)} = \left\{ {\begin{matrix}{\frac{K}{H^{2}\left\lbrack {1 + {\tan^{2}\left( \psi^{i} \right)}} \right\rbrack},} & {q \in {F_{i}\left( {{cx}^{i},{cy}^{i},\theta^{i},\psi^{i}} \right)}} \\{0,} & {otherwise}\end{matrix},} \right.} & (8)\end{matrix}$where K is a sensor constant that depends on the number of pixels in anacquired image. If all of the sensors have the same value of K, then thevalue is unimportant for the optimization described below.

At any time j, the objective function we minimize is a differencebetween the desired resolution and an achieved resolution up to time j−1according to the following function:

$\begin{matrix}{{{G_{j}\left( {x,\theta,\psi} \right)} = {\int_{Q}{{f\left( {{\varphi_{d}(q)} - \left\lbrack {{\varphi_{j - 1}^{p}(q)} + {\sum\limits_{i}\;\left( {R^{i}\left( {{cx}^{i},{cy}^{i},\theta^{i},\psi^{i},q} \right)} \right)^{p}}} \right\rbrack^{1/p}} \right)}\ {\mathbb{d}q}}}},} & (9)\end{matrix}$where φ_(j−1)(q) is the resolution achieved by the sensors up to timej−1, p defines the norm used to model a subadditive combination ofoverlapping footprints, and f(x) is a penalty function that penalizesdeviation from the desired resolution.

For example, in one embodiment, f(x)=x². This penalty function penalizesthe achieved resolution when the resolution is lower or greater than thedesired resolution. This forces the sensors to move to a different areaof the environment when some of the sensors have been mapped to asufficient resolution.

In another embodiment,

${f(x)} = \left\{ {\begin{matrix}{x^{2},} & {x \geq 0} \\{0,} & {x < 0}\end{matrix}.} \right.$

This penalty function penalizes the achieved resolution only when it hasnot attained the desired resolution, which enables the sensor tocontinue improving the resolution of the imaged area beyond thepre-specified desired resolution. Of course, other embodiments may useother penalty functions.

Path Planning

FIG. 6 shows a method for path planning according to embodiments of theinvention. By definition, the initial achieved resolution φ₀(q) isidentically zero.

A gradient-based optimization is described by the followinginitialization and iterative steps. At each time j, a completegradient-based minimization with respect to the angle parameters of thesensors is performed. However, sensor positions are updated using only asingle gradient step. The reason is that after the sensors have movedand acquired new data, the objective function has changed.

Initialization

Given the desired resolution φ_(d)(q), and a vector x₀ of initial sensorpositions, initial sensor angles are determined 605 by the followingoptimization 611

$\begin{matrix}{\theta_{0},{\psi_{0} = {\arg\;{\min\limits_{\theta,\psi}\;{{G_{0}\left( {x_{0},\theta,\psi} \right)}.}}}}} & (11)\end{matrix}$

The initial position gradient g₀ is the gradient with respect to x ofG₀(x, θ₀, ψ₀) evaluated at x₀.

Iteration 650 j=1, 2, . . . at each Sensor for each Time Step

Acquire 610 images 601 from all sensors 100, and update 620 an achievedresolution 621 according to

$\begin{matrix}{{\varphi_{j}\left( {q,x_{j - 1},\theta_{j - 1},\psi_{j - 1}} \right)} = {\left\lbrack {{\varphi_{j - 1}^{p}(q)} + \left( {\sum\limits_{i}\;\left( {R^{i}\left( {{cx}_{j - 1}^{i},{cy}_{j - 1}^{i},\theta_{j - 1}^{i},\psi_{j - 1}^{i},q} \right)} \right)^{p}} \right)} \right\rbrack^{1/p}.}} & (12)\end{matrix}$

Moving 630 the set of sensors to a next position and orientation of thesensor based on the achieved resolution and the desired resolution. Themoving is in a direction of the negative position gradient 631x _(j) =x _(j−1) −αg _(j−1)  (13)where α is a step size, and g_(j−1) is the position gradient at aprevious time evaluated at position x_(j−1). It should be understoodthat the moving to the next position and orientation can be a “null”move, i.e., the sensor remains in place.

Update 640 the sensor angular parameters and the position gradient 641according to

$\begin{matrix}{\mspace{79mu}{\theta_{j},{\psi_{j} = {\arg\;{\min\limits_{\theta,\psi}\;{G_{j}\left( {x_{j},\theta,\psi} \right)}}}}}} & (14) \\{{g_{j} = {{gradient}\mspace{14mu}{with}\mspace{14mu}{respect}\mspace{14mu}{to}\mspace{14mu} x\mspace{14mu}{of}\mspace{14mu}{G_{j}\left( {x,\theta_{j},\psi_{j}} \right)}\mspace{14mu}{evaluated}\mspace{14mu}{at}\mspace{14mu} x_{3}}},} & \;\end{matrix}$and iterate for the next time instant j=j+1.

Although the invention has been described by way of examples ofpreferred embodiments, it is to be understood that various otheradaptations and modifications can be made within the spirit and scope ofthe invention. Therefore, it is the object of the appended claims tocover all such variations and modifications as come within the truespirit and scope of the invention.

We claim:
 1. A method for planning paths of a set of sensors in anenvironment, wherein the sensors are mobile sensors with changeablepositions and orientations, and each sensor includes a processor, animaging system and a communication system, comprising the steps of:defining a desired resolution of coverage of the environment;initializing an achieved resolution of the coverage, and furthercomprising for each time instant in each processor of each sensor thesteps of: acquiring an image of the environment using the imagingsystem; updating the achieved resolution according to the image; andmoving to a next position and orientation based on the achievedresolution and the desired resolution; and distributing localinformation of the sensor to other sensors in the set of sensors usingthe communication system to optimize a coverage of the environment. 2.The method of claim 1, wherein the imaging system uses optical imaging.3. The method of claim 1, wherein the imaging system uses syntheticaperture radar.
 4. The method of claim 1, wherein the imaging systemuses hyperspectral imaging.
 5. The method of claim 1, wherein theimaging system uses physical aperture radar imaging.
 6. The method ofclaim 1, wherein information comprises of the position, the orientationand the achieved resolution.
 7. The method of claim 1, furthercomprising: arranging the sensor in an indoor drone, an outdoor drone,an airplane, a statelitte, a ground vehicle, a robot, or an underwatervehicle.
 8. The method of claim 1, wherein the communication systemincludes wireless channels.
 9. The method of claim 1, wherein thepositions are updated at a different rate than the orientations.
 10. Themethod of claim 1, wherein a model of the resolution is subadditive. 11.The method of claim 1, further comprising: minimizing a differencebetween the desired resolution and the achieved resolution using anobjective function.
 12. The method of claim 11, wherein the objectivefunction uses a function that penalizes a deviation from the desiredresolution.
 13. The method of claim 12, wherein the function penalizesthe achieved resolution when the resolution is lower or greater than thedesired resolution.
 14. The method of claim 12, wherein the functionpenalizes the achieved resolution only when the achieved resolution hasnot attained the desired resolution.
 15. The method of claim 1, whereinthe moving is in a direction of a negative position gradient.